Requirements for resource management game AI

This paper examines the principles that define resource management games, popular and challenging constructive computer games such as SIMCITY and VIRTUAL U. From these principles, it is possible to derive requirements for intelligent programs designed to play such games, as a replacement of a human player. A first step for research in the domain of intelligent programs playing resource management games is presented, in the form of a hybrid AI approach that combines abductive planning with evolutionary learning. The results obtained by this approach are promising. 1 Artificial intelligence in interactive computer games One of the main goals of AI research always has been the development of artificial intelligence that is as versatile as human intelligence. For many years, the game of CHESS has been the drosophila melanogaster of artificial intelligence research [McCarthy, 1990], but in the last decades, it is argued that games such as CHESS are not able to address all abilities of intelligence sufficiently, because they are abstract and completely deterministic [Pfeiffer and Scheier, 1999; Laird and van Lent, 2001]. Computers are good at calculation and therefore inherently good at dealing with abstract and deterministic problems, but being good at calculation clearly is not the same as being able to make intelligent decisions [Pfeiffer and Scheier, 1999]. In recent years, the computer games industry has received increasing attention from the artificial intelligence community, because interactive computer games, while being a closed world, can require a computer to perform tasks that currently only humans are able to perform sufficiently well. Interactive computer games are therefore seen as an ideal testbed for alleged computer intelligence [Laird and van Lent, 2001]. Potential tasks for AI in computer games can vary from entertaining the human player to actually being able to play an entire game. Since designers of interactive computer games aim at a commercial rather than a scientific goal, computer controlled players for such games are rarely, if ever, designed to explore the game autonomously to arrive at highquality decision making capabilities (i.e., high-quality game AI). Therefore, if game AI is applied in practise, it is often static and only encompasses the designers’ insight on what the game constitutes. We observe three problems here. First of all, from a scientific perspective, we should aim at developing AI methods that do not need designers’ insight in order to work well. A method that works well because it reproduces knowledge programmed ahead of time, may be able to play an intelligent game, but cannot be considered to be intelligent itself. Moreover, human designers may make mistakes, or fail to take into account relevant information, which makes the game AI of inferior quality. Second, methods using knowledge programmed ahead of time will not be able to adapt when a game’s parameters change, or when opponents are encountered that follow strategies not taken into account. In commercial computer games, the specifications are often changed to resolve balance issues. Adaptive AI methods offer an advantage here: if changes are needed, the methods can adapt easily and do not need to be rebuilt. Third, even if our only goal is to develop computercontrolled players for commercial games, we must realize that most human players expect that a computer plays fairly (i.e., it does not use knowledge that human players do not have) [Scott, 2002]. For example, Garry Kasparov was furious with Deep Blue when the computer had beaten him in CHESS, because the grandmaster was convinced that it had been using information provided during the game by a team of CHESS players [Jayanti, 2003]. In this paper, we examine the principles that define resource management games, which are interactive computer games with a constructive nature. These principles lead to a list of requirements for intelligent programs designed to play such games, as a replacement of a human player. Exploratory research in the domain of intelligent programs playing resource management games shows that hybrid AI approaches, combining abductive planning with evolutionary learning, are a possible way of dealing with these requirements. In section 2 of this paper, we address the relevance of research in the domain of resource management games. In section 3, we discuss the principles underlying these games and derive the required capabilities of resource management game players (including computercontrolled players). Section 4 presents a possible solution method that is able to deal with these capabilities, and section 5 continues with a brief overview of experiments that compare the performance of this method with more conventional methods. In section 6, we conclude and look at future work. 2 Relevance of research into resource management games We define resource management games as interactive computer games where the main problem for the player1 is to use limited resources to construct and maintain a complex virtual environment. These games have proved to be challenging and entertaining, because of their many layers of complexity, arising from a rather small and understandable set of actions. They usually require a player to construct buildings and move units in a large grid world. While playing a resource management game, the player tries to reach a certain goal by carefully distributing limited resources. A famous example is the game SIMCITY, in which the player builds and manages a city. Resource management games share many ideas with strategy games such as WARCRAFT [Laird and van Lent, 2001; Buro, 2004; Ponsen and Spronck, 2004], but their nature and game play differ significantly from these games, as explained below. The main difference in nature is that in strategy games, the player constructs buildings and controls units of a virtual army in order to defeat opponent armies, whereas in resource management games, the focus is on the construction and long-term maintenance of buildings, transport networks, et cetera. Due to this difference in nature, strategy games are in use by many military organizations as a source of inspiration [Laird and van Lent, 2001], whereas resource management games receive more attention from economists and managers [Sawyer, 2002]. One might summarize that strategy games have a destructive nature, whereas resource management games have a constructive nature. One of the two main differences in game play between the genres is the fact that strategy games are always finite games – once the enemy has been defeated, the game ends – whereas resource management games do not need to be finite, with goals such as ‘build and maintain a large city’ or ‘transport many passengers’. A second difference in game play is that most strategy games progress in Henceforth, we will use the term ‘player’ to indicate both a human player and a computer-controlled player, unless indicated otherwise. near-continuous time, hence the name real-time strategy (RTS) games [Ponsen and Spronck, 2004], whereas most resource management games progress in clearly observable discrete time. Near-continuous games, such as realtime strategy games, enable all players to move at the same time, which entails that players must possess both the capacity to respond quickly to urgent matters and the capacity to think about strategic problems. Discrete or turn-based games, such as resource management games, are similar in pace to games such as CHESS: each player takes his turn to perform a limited number of actions, and is allowed to think about which actions to perform. Because of this, the challenges posed by discrete games often require more structured thinking and less intuitive response than those posed by near-continuous games – for example, a player has to construct a sound planning towards some end goal. There are three reasons why it is relevant to perform research in the area of AI for computer-controlled players for resource management games. First, because of the many challenges involved in playing interactive computer games, a computer-controlled player for such games must be able to deal with many aspects of humanlevel intelligence [Laird and van Lent, 2001]. Resource management games are able to address aspects such as high-level planning, which are not often found in other genres of interactive computer games, but rather in classical board games such as CHESS. Therefore, resource management games can be said to bridge the gap between classical games and interactive computer games. Second, problems found in many other domains (such as other genres of computer games, management and economics) closely resemble problems typically found in resource management games. Third, resource management games are being developed not only as pure entertainment, but also as educative tools. For example, Harvard’s Business School is known to use resource management games in its student training program [Sawyer, 2002]. AI techniques that are able to deal with educative resource management games such as VIRTUAL U can be valuable sources of strategic information for scientists and students; in other words, the solutions AI techniques come up with, can be analysed and used as inspiration for people playing these games. 3 Game principles and players’ capacities A typical resource management game is played in a simulated world, usually a grid that the player looks at in bird’s eye view. As time passes, a player and/or the game itself can place various kinds of structures on the grid, for example roads and buildings. The content of the grid defines the state of the game world, which is internally represented by a set of parameters. The game’s dynamics are defined by a set of actions that determine the transition from the current state of the game into a new state. Both the game itself and the player can execute these actions. When playing the game, the player has to cope with many temporary objectives that eventually enable him to reach the end goals of the game. Playing a resource management game requires thorough planning: the player must find out which actions to perform in order to eventually reach the end goals of the game, given the current state of the game world. To be able to do this, he must take into account the fact that resources and dynamical aspects play an important role2. Resource management games use a set of principles that lead to challenging and surprising game play for the player. We will illustrate these principles with a basic resource management game called FACTORY, in which the player can build houses and factories in order to control unemployment rates in a city. Figure 1 illustrates a possible state and interface for this game. The goal of the game is to fill the entire grid and reach a stable unemployment rate of 0%. A player can place houses and factories on the grid using the buttons BUILD HOUSE and BUILD FACTORY. The game’s response to placing objects on the grid is a new value for the player’s money and the current unemployment rate in the city. The game’s actual progress is determined by one or more of the principles outlined below. If we are aiming at developing a computer-controlled player for resource management games, we must keep in mind that such a player must be able to address all of these principles. 1. Dependence on resources – Every resource management game contains this principle, which entails that resources are limited and that there are dependencies between some of them. If for example we build one factory, for which we require 100,000 euros, at most 100 unemployed people will find a job and will start paying taxes. 2. Dependence on location – This principle implies that the location on the grid where we build objects influences the effect on the game world. For example, if we build a factory too close to a residential area, people will start complaining about pollution and decide to stop paying taxes, whereas if we build the factory too far from the residential area, people will refuse to go to work. 3. Dependence on time – This principle implies that in some cases, the effect of a certain action is delayed. For example, if we build a factory now, it will be finished in 12 months (game time). 4. Need for planning – A player cannot execute all possible actions immediately from the beginning of the game. For example, the player must gather at least 100,000 euros before a factory can be built. Furthermore, the game rules might stipulate that a factory can only be built if there are sufficient people available. These people must have housing, In this respect, the planning problems encountered in resource management games differ substantially from classical planning problems, that are successfully addressed by various symbolic AI techniques. In these classical planning problems, the focus is on state transitions, usually in a deterministic or even static environment without competitors. which means we can only build a factory after we have built at least a few houses for the workers to live in. Thus, if the player wants to build a factory, he must come up with a sequence of actions that makes the game progress from the current state to a state in which building a factory is possible. 5. Competition – In many resource management games, the player has to compete with computercontrolled opponents or co-operate with allies, sharing limited resources effectively. 6. Multiple objectives – Resource management games often require the player to pursue more than one objective at once in order to reach the end goals of the game. For example, in the FACTORY game, the player will be trying to build as many factories as possible, but also to keep his financial situation healthy and to outperform any competition, and he does all this to reach the game’s goal: creating a city that has an unemployment rate of 0%. 7. Non-determinism – Many games include elements of non-determinism. Practical implementations vary from adding odd-behaving competitors or natural disasters to a game to introducing noise on the effects of rules that specify game dynamics. For example, if we build a factory, we expect 100 people to find a job, but the actual number of people that do find a job varies. From these seven principles, we derive the following required capabilities for a player of resource management games. He must be able (1) to cope with resource dependencies, (2) to allocate space effectively, (3) to predict future game states, (4) to create a sound planning, (5) to handle competition , (6) to perform multi-objective reasoning, and (7) to deal with non-determinism. All of these capabilities are actually used for only one task: selecting actions that should be executed. In deterministic games, the sequence of actions selected by each of the players defines the outcome completely. In nondeterministic games, the sequence of actions also plays an important role in the outcome – after all, if the outcome is determined more by randomness than by the actions executed, the game is more a game of chance than a real resource management game. Thus, action selection is the core task of any resource management player, including computer-controlled players.